REPORT DOCUMENTATION PAGE 


Public reporting burden for this collection of information is estimated to average 1 hour per response, inc 
gathering and maintaining the data needed, and completing and reviewing the collection of information, 
collection of information, including suggestions for reducing this burden, to Washington Headquarters Set 
Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, P 

1. AGENCY USE ONLY (Leave blank) I 2. REPORT DATE 13. RE 


September 2000 


4. TITLE AND SUBTITLE 


Stochastic Set Partitioning Methods for Operational Planning of Aircraft 


AFRL-SR-BL-TR-01 - 

1 January 1993 - 31 December 1995 


5. FUNDING NUMBERS 

F49620-93-1-0098 


6. AUTHOR(S) 
Warren B. Powell 


7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

Princeton University 

Department of Operations Research and Financial Engineering 
Princeton, NJ 08544 


8. PERFORMING ORGANIZATION 
REPORT NUMBER 


9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

AFOSR 

801 North Randolph Street, Room 732 
Arlington, VA 22203-1977 


10. SPONSORING/MONITORING 
AGENCY REPORT NUMBER 

F49620-93-1-0098 



12a. DISTRIBUTION AVAILABILITY STATEMENT 

Approved for Public Release, Unlimited Distribution 


12b. DISTRIBUTION CODE 


13. ABSTRACT (Maximum 200 words) 

In the last two years, the project has focused on developing tools for representing, mathematically and in software, complex 
operational problems. We have developed a formal problem class called Dynamic Resource Transformation Problems which 
captures in a compact way a broad range of complex applications. We have developed a simulation library that captures the 
specific features of this problem class, in particular the ability to make decisions within the simulation. Finally, we have 
developed an algorithmic metastrategy that provides an effective way of solving these problems. We consider this work to be 
an important breakthrough in our ability to model and solve the types of operational challenges that arise in military airlift 
operations, as well as a variety of civilian applications such as railroads and trucking, as well as air traffic control. 


20010301 069 



15. NUMBER OF PAGES 


16. PRICE CODE 


17. SECURITY CLASSIFICATION 18. SECURITY CLASSIFICATION 19. SECURITY CLASSIFICATION 20. LIMITATION OF ABSTRAC 
OF REPORT OF THIS PAGE OF ABSTRACT 


Standard Form 298 (Rev. 2-89 EG 

Prescribed by ANSI Std. 239.18 

Designed using Perform Pro, WHS/DIOR, Oct 94 




















Final Report: 


Stochastic Set Partitioning Methods for Operational Planning of Aircraft 

Principal Investigator: 

Warren B. Powell 

Grant: F49620-93-1-0098 

September, 2000 


Princeton University 

Department of Operations Research and Financial Engineering 
Princeton, NJ 08544 
powell@princeton.edu 




Abstract 


In the last two years, the project has focused on developing tools for representing, 
mathematically and in software, complex operational problems. We have developed a 
formal problem class called Dynamic Resource Transformation Problems which captures 
in a compact way a broad range of complex applications. We have developed a 
simulation library that captures the specific features of this problem class, in particular 
the ability to make decisions within the simulation. Finally, we have developed an 
algorithmic metastrategy that provides an effective way of solving these problems. 


We consider this work to be an important breakthrough in our ability to model and solve 
the types of operational challenges that arise in military airlift operations, as well as a 
variety of civilian applications such as railroads and trucking, as well as air traffic control. 



Progress report-2000 


Page 1 


1. Objectives 

We are developing a new modeling environment for a broad problem class called 
Dynamic Resource Transformation Problems (or DRTP’s). Specific subtasks include: 

Task 1) Development of a general representational framework that captures all 
the dimensions of a DRTP in a compact way. 

Task 2) Development of a flexible mathematical notation that captures the 
structure of this class of problems. Our goal here is a simple, flexible notation 
that can easily reflect a broad range of complex operational issues (such as 
those studied by AMC using MASS) and yet still retain the structure of the 
problem so that we can optimize it (in contrast with MASS, which is a pure 
simulation). 

Task 3) Development of a flexible software object library that allows us to 
execute the equations developed with the notation system. Our goal here is an 
architecture that easily handles a high level of complex operational issues yet 
maintains a separation between the optimization algorithm and the underlying 
model. 

Task 4) Demonstration of our environment using the Airlift Flow Model as a 
demonstration. 

Task 5) Design of algorithms that will produce optimal or near-optimal solutions 
to problems such as the AFM. 

Task 6) Development of gradient estimates that will tell us what are the 
constraining resources at the end of a simulation. 


2. Status of effort 

Our progress in each of the tasks listed above: 

Tasks 1 and 2) We have recently completed a representational paradigm that 
describes the fundamental dimensions of a DRTP, and provides an elegant 
mathematical representation. This model is being tested through its 
application, in three separate projects using different people, to a variety of 
operational problems. 

Task 3) We have a prototype version of the simulation library. This is fully 
documented using a private web page, but we can provide access to any 
interested party. 

Task 4) We have begun developing our own version of the Airlift Flow Model 
using data provided by AMC at Scott AFB. We do not intend our model to 
have all the details of the production version, but we want to have sufficient 




Progress report-2000 


Page 2 


realism for us to demonstrate proof of concept, and also to investigate new 
optimization strategies that could be put into the production system. 

Task 5) We continue to make progress on algorithms for solving this problem 
class. This past year, we made progress in a particular problem class which 
involves stochastic, dynamic resource allocation problems in the presence of 
multiperiod travel times. All transportation problems exhibit multiperiod 
travel times, but this presents particular modeling challenges in a stochastic 
setting. 

Task 6) We have only begun research into gradient estimates from a simulation 
run. 


3. Accomplishments/new findings 

1. Representation of DRTP’s: The conceptual and mathematical representation 
of DRTP’s has given us a powerful new vocabulary that extends optimization 
into a dynamic arena, and provides for optimization logic within a simulation 
setting. 

2. Java-based simulation library for DRTP’s: A major result over the last 
several years has been the development of a new library for representing 
DRTP’s in software. The implementation closely follows the DRTP paradigm 
that we have recently completed. 

3. Dynamic programming approximations for resource allocation problems: 

We have developed a new class of dynamic programming approximations for 
resource allocation problems based on the adaptive estimation of piecewise 
linear functions using the CAVE algorithm. The algorithm appears to give us 
optimal solutions in certain problem classes, and very good solutions in more 
general problems. Convergence is fast and stable. 

4. Stochastic allocation problems in the presence of multiperiod travel 
times: The challenge of solving resource allocation problems under 
uncertainty in the presence of multiperiod travel times has been largely 
overlooked, and yet introduces serious errors in many settings. For example, 
we may allow an unimportant aircraft to occupy airbase capacity in a 
congested period just because it took longer to get there, and hence the 
decision was made farther in the future. We have developed an elegant 
solution that produces near-optimal results when tested in a deterministic 
environment (for which optimal solutions are available). Solutions are always 
integer. 

5. Stochastic allocation problems in the presence of multiperiod travel 
times: The challenge of solving resource allocation problems under 
uncertainty in the presence of multiperiod travel times has been largely 
overlooked, and yet introduces serious errors in many settings. For example, 
we may allow an unimportant aircraft to occupy airbase capacity in a 
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congested period just because it took longer to get there, and hence the 
decision was made farther in the future. We have developed an elegant 
solution that produces near-optimal results when tested in a deterministic 
environment (for which optimal solutions are available). Solutions are always 
integer. 

6. Pattern matching behaviors in problem representation: We are developing 
a pattern-based approach for improving the acceptability of solutions without 
extensive engineering. This has already proven valuable for our work with 
Yellow Freight System and Norfolk Southern Railroad, and we are continuing 
to use this powerful concept in our other work. 


4. Personnel supported 

Faculty: 

• Professor Warren B. Powell 
Research staff: 

• Dr. Jack Gelfand, Senior Research Scientist 
Graduate students: 

• Greg Godfrey 

• Joel Shapiro 

• Huseyin Topaloglu 

• Mike Spivey 

• Katarina Papadaki 
Undergraduates 

• Steve Woolbert 

5. Publications (10/1/97 - 9/30/98) 

5.1. Submitted but not yet accepted: 

“Network Representation, Column Generation and Branch and Bound for Parallel Machine 
Scheduling with Multiple Job Families,” (with Zhi-Long Chen) Submitted to Operations Research, 
March, 1998. 

"Adaptive Estimation of Daily Demands with Complex Calendar Effects," submitted to 
Transportation Research (with G. Godfrey). September, 1997. 



Progress report-2000 


Page 4 


"A Multiplier Adjustment Method for Dynamic Resource Allocation Problems," submitted to 
Transportation Science (with Tassio Carvalho). Under revision, July, 1998. 

Powell, W.B. and J. Shapiro, ''A Dynamic Programming Approximation for a REALLY BIG 
Large-Scale Driver Scheduling Problem," submitted to Operations Research. March. 1998. 

Godfrey, G. and W.B. Powell, "An Adaptive Approximation Method for Stochastic, Dynamic 
Programs, with Applications to Inventory and Distribution Problems," submitted to Management 
Science, July, 1997. 

Chen, Z.-L. and W.B. Powell, "A Convergent Cutting Plane and Partial Sampling Algorithm for 
Multistage Linear Programs with Recourse," submitted to JOTA. 

"Adaptive Labeling Algorithms for the Dynamic Assignment Problem," submitted to 
Transportation Science (with W. Snow and R. Cheung). Under revision for resubmission. 

“A Representational Paradigm for Discrete Dynamic Resource Transformation Problems," 
submitted to Annals of Operations Research on Modeling (Fourer, Gay and Geoffrion, eds), 
March, 1998. 

J.J. Gelfand, S.L. Epstein and W.B. Powell, “Integrating Pattern Learning in Multimodal Decision 
Systems,” 

G. Godfrey and W.3. Powell, “Control of Multistage, Stochastic, Dynamic Resource Allocation 
Problems,” submitted to Operations Research, under revision. 

J. Shapiro and W.B. Powell, “An Object-Oriented Framework for Dynamic Resource 
Transformation Problems,” submitted to Informs Journal on Computing, September, 1998. 


5.2. Accepted but not yet published: 

Powell, W.B., M.T. Towns and A. Marar, “On the Value of Globally Optimal Solutions for 
Dynamic Routing and Scheduling Problems,” Transportation Science, To appear. 

R. K.-L. Cheung and W.B. Powell, "The Convergence of Hybrid Stochastic Gradient Methods in 
Stochastic Programming, with an Application to Dynamic Networks with Random Arc Capacities," 
Operations Research. To appear. 

"A Column-Generation Based Decomposition Algorithm for a Parallel Machine Just-In-Time 
Scheduling Problem," European Journal of Operations Research (with Zhi-Long Chen). March, 
1997. To appear. 

"Solving Parallel Machine Scheduling Problems by Column Generation," Informs Journal of 
Computing (with Zhi-Long Chen). To appear. 


5.3. Published: 

Powell, W.B. “On Languages for Dynamic Resource Scheduling Problems,” in Fleet Management 
and Logistics, (T. G. Crainic and G. Laporte, eds.), Kluwer Academic Publishers, Boston, 1998. 

"A Generalized Threshold Algorithm for the Shortest Path Problem with Time Windows," in 
Network Design: Connectivity and Facilities (P. Pardalos, D. Du, eds.), pp. 303-318 (1998). (with 
Zhi-Long Chen). 
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"An Optimal Control Formulation of Large-Scale Multiclass Machine Scheduling Problems," in 
Network Optimization , Lecture Notes in Economics and Mathematical Systems, (P. Pardalos, D. 
Hearn and W. Hager, eds.), Springer-Verlag, New York, pp. 423-440, 1997 (with Z.-L. Chen). 

"Dynamic Control of Logistics Queueing Networks for Large Scale Fleet Management," 
Transportation Science, Vol. 32, No. 2, pp. 90-109, 1998. 

Powell, W.B. and T. Carvalho, "Real-Time Optimization of Containers and Flatcars for Intermodal 
Operations," Transportation Science, Vol. 32, No. 2, pp. 110-126, 1998. 

"Finding the Yellow Brick Road: Part VII: I Finally Have a Brain!," Interfaces , Vol. 27, No. 5, 
pp. 9-14. (with Don Mayoras). 

"Finding the Yellow Brick Road: Part VI: Courage! 44 Interfaces , Vol. 27, No. 4, pp. 12-22 (1997). 
(with Don Mayoras) 

"Finding the Yellow Brick Road: Part V: Through the Crystal Ball 44 Interfaces , Vol. 27, No. 3, pp. 
14-21 (1997). (with Don Mayoras) 

"Dynamic Control of Multicommodity Logistics Queueing Networks," European Journal of 
Operations Research, Vol. 98, No. 3, 1997. (with Tassio Carvalho). 

"A Note on Bertsekas' Small-Label-First Strategy," Networks, Vol. 29, No. 2, pp. 111-116 (with 
Zhi-Long Chen). 


5.4 Technical reports 

1 "A Generalized Threshold Algorithm for the Shortest Path Problem with Time Windows," 
submitted to Journal on Computing, (with Zhi-Long Chen). 


6. Interactions/transitions 


6.1. Participation/presentations at meetings, conferences, etc. 

Invited talks: 

“Approximate Methods for Multistage Dynamic Programs for Discrete Dynamic Resource 
Allocation Problems,” Stochastic Programming Conference, Vancouver, BC, August, 1998 (with 
G. Godfrey). 

“A Convergent Cutting Plane Algorithm for Multistage Stochastic Programs,” Stochastic 
Programming Conference, Vancouver, BC, August, 1998 (with Z.-L. Chen). 

“Models, Formulations and Algorithms for Multi-Layered Resource Scheduling Problems,” Air 
Products and Chemicals, Allentown, PA, July, 1998. 

“Locomotive Scheduling as a Dynamic Resource Transformation Problem,” Research conference, 
Norfolk Southern Railroad, Atlanta, GA, July, 1998. 

Invited tutorial: “A Representational Paradigm for Dynamic Resource Transformation Problems,” 
Triennial Symposium on Transportation Analysis III, Puerto Rico, June, 1998. 
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“Approximate Methods for Dynamic Resource Allocation Problems,” Triennial Symposium on 
Transportation Analysis III, Puerto Rico, June, 1998 (with G. Godfrey). 

“Implementation of a Dynamic Fleet Management System,” Triennial Symposium on 
Transportation Analysis III, Puerto Rico, June, 1998 (with A. Marar). 

“Dynamic Resource Transformation Problems: Problem Representation and Solution 
Algorithms”, New World Vistas Conference, sponsored by Air Force Office of Scientific 
Research, Monterey, CA, May, 1998. 

“Dynamic Resource Transformation Problems: A New Modeling Approach for Air Mobility 
Problems,” Air Mobility Users Group Conference, Air Force Academy, Colorado Springs, CO, 
May, 1998. 

“Management Issues in Transportation Modeling and Optimization,” Freight Transportation Short 
Course, Georgia Institute of Technology, Atlanta, Georgia, August, 1997. 

“Control of Dynamic Resource Transformation Problems: An Overview of CASTLE Laboratory ,” 
Airlift Mobility Command, St. Louis, MO, July, 1997. 


Conferences: Numerous talks at the biannual INFORMS meetings. 


6.2. Consultative and advisory functions 

I regularly meet with the group at the Airlift Mobility Command, both at Scott AFB and 
one or twice a year at their air mobility group users meetings. We have begun developing 
our own version of the airlift flow module so that we can work directly with their data, 
and thereby better understand the specific issues they are grappling with. This simulator 
will allow us to try out ideas that might be implementable in the production version of the 
code. 


6.3. Transitions 

A major thrust of my research program is the development of general purpose analysis 
tools for dynamic resource management problems. This approach allows us to identify 
opportunities for applications of our developments in different arenas, although our focus 
is entirely on transportation problems similar to those faced by AMC. 

Specific transitions that have occurred include: 


1 Transition: We have applied our optimal control methodology based on our 
new “logistics queueing network” (LQN) formulation, to the management of 
their intermodal flatcar fleet. 

Recipient: Norfolk Southern Railroad and Burlington Northern Sante Fe 
Railroad. 




Progress report-2000 


Page 7 


2 Transition: We have applied our optimal control methodology to the 
management of the linehaul network for LTL motor carriers. These problems 
involve tactical planning of over 6,000 drivers. 

Recipient: Yellow Freight System, Inc. 

3. Transition: We have adapted our multiattribute labeling algorithm to handle 
real-time load planning for truckload motor carriers. 

Recipient: Burlington Motor Carriers 

4. Transition: Adaptive estimation of exogenous information events. This past 
year we produced a major refinement of a hierarchical exogenous activity 
estimation system. This year (1998) it went into production to help Yellow- 
Freight System forecast shipper activity and network performance. 

Recipient: Yellow Freight System 

5. Transition: An algorithm for multi-layered dynamic resource transformation 
problems. 

Recipient: Air Products and Chemicals 

Last year saw the first wave of transitions by Transport Dynamics which is working with 
technology produced by CASTLE Lab, licensed from Princeton University. Two of these 
transitions include: 

6. Transition: Driver scheduling and capacity forecasting in a stochastic 
environment for a fast-response motor carrier. This system was licensed by 
the University to Transport Dynamics. 

Recipient: Roberts Express, implemented by Transport Dynamics under 
license from Princeton University. 

7. Transition: Dynamic load planning for less-than-truckload motor carriers 

Recipient: Roadway Package System, implemented by Transport Dynamics 
under license from Princeton University. 


7. New discoveries, inventions or patent disclosures 

This fall we will make a formal disclosure of our new simulation library, developed in 
Java, for simulating dynamic resource transformation problems. 
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Our research has produced six awards for graduate student research. Last year we learned 
of: 

Winner, 1997 Transportation Dissertation Prize Competition, won by Tassio Carvalho. 
Dissertation title: Dynamic Control of Spatial Resource Allocation Problems." 



